1. Field of the Invention
The present invention relates to a luminance signal generation apparatus, a luminance signal generation method, and an image capturing apparatus. The present invention particularly relates to an apparatus that generates a luminance signal from a signal obtained by an image sensor that uses a primary color Bayer pattern type of color filter array, a method for the same, and an image capturing apparatus that uses the apparatus and method.
2. Description of the Related Art
Since a color image is generated with use of an image sensor capable of detecting light quantities, such as a CCD image sensor or a CMOS image sensor, a configuration in which light is passed through a color filter array and then caused to incident on the image sensor is generally used.
There are various types of color filter arrays that differ according to colors used therein, the pattern of colors allocated to pixels, and the like, and primary colors (red, green, and blue) or complimentary colors (cyan, magenta, and yellow) are widely-used color types, and the Bayer pattern is widely used as the color pattern.
FIG. 20 is a diagram showing an exemplary configuration of a luminance signal generation circuit that realizes an OutOfGreen method for generating a luminance signal (OG signal) from only a green (G) signal, which is one of various conventional methods for generating a luminance signal with use of a primary color Bayer pattern type of color filter array. R represents red, G1 and G2 represent green, and B represents blue.
First, a 0-insertion circuit 2201 is applied to a RAW signal 2200 obtained by digitizing output (2204) from an image sensor, thereby setting the values of pixels other than the G pixels to 0 (2205). Next, a low-pass filter (V-LPF) circuit 2202 that limits the bandwidth in the vertical direction and a low-pass filter (H-LPF) circuit 2203 that limits the bandwidth in the horizontal direction are applied to obtain a luminance signal. Hereinafter, a luminance signal obtained using the OutOfGreen method is called an OG signal.
Another example of a conventional method for generating a luminance signal with use of a primary color Bayer pattern type of color filter array is an SWY method in which the signals of all RGB pixels are handled equally as a luminance signal (SWY signal).
FIG. 21 is a diagram showing an exemplary configuration of a luminance signal generation circuit that realizes the SWY method.
As is clear from a comparison with FIG. 20, the SWY method is a method of obtaining a luminance signal without using the 0-insertion circuit 2201 in the OutOfGreen method. Hereinafter, a luminance signal obtained using the SWY method is called an SWY signal.
FIG. 22 is a diagram showing the resolvable spatial frequency characteristics of an OG signal and an SWY signal.
The x-axis indicates the frequency space of an imaging subject in the horizontal (H) direction, the y-axis indicates the frequency space in the vertical (V) direction, and the spatial frequency increases with increasing distance from the origin point. In the case of an OG signal, a luminance signal is obtained from only the G signal, and therefore the resolution limit in the horizontal and vertical directions is equal to the Nyquist frequency of the image sensor (on the axis, π/2). However, since there are lines in which pixels do not exist in the diagonal directions, the limiting resolution frequency in the diagonal directions is lower than in the horizontal and vertical directions, and as a result, a diamond-shaped spatial frequency region 2400 is the resolvable spatial frequency.
On the other hand, in the case of an SWY signal, a signal is obtained using all pixels, and therefore when the imaging subject is achromatic, a square region 2401 as shown in FIG. 22 is the resolvable spatial frequency. However, in the exemplary case of a red imaging subject, a luminance signal is not output from pixels other than R pixels, and therefore resolution is only possible in a spatial frequency range 2402 that, when compared with an achromatic imaging subject, is half the range in the horizontal and vertical directions.
Japanese Patent Laid-Open No. 2003-348609 proposes a method in which diagonal regions 2403 of the OG signal in FIG. 22 are substituted with an SWY signal. However, since the resolution limit frequency of an SWY signal falls when the imaging subject is chromatic, the OG signal is replaced with the SWY signal only if the diagonal regions 2403 pertain to an achromatic image subject. Thereafter, an edge emphasis component is detected with use of the generated luminance signal and added to the luminance signal, thus generating a final luminance signal.
Japanese Patent Laid-Open No. 2003-196649 proposes a method in which a plurality of pre-provided interpolation filters are used differently depending on the angle of the imaging subject when generating a luminance signal, and then the luminance signal is edge-emphasized, thus generating a final luminance signal.
Also Japanese Patent No. 3699873 proposes a method in which, similarly to Japanese Patent Laid-Open No. 2003-348609, weighted-addition is performed on an OG signal and an SWY signal according to the hue and chroma of the imaging subject. Specifically, an SWY signal is used for a low-chroma imaging subject, an SWY signal is used for a high-chroma imaging subject as well as an Mg (magenta) and a G (green) imaging subject, and an OG signal is used for other chromatic imaging subjects. Thereafter, an edge emphasis component is computed with use of a MIX signal and added to a separately generated luminance signal, thus generating a final luminance signal.
Furthermore, in a method disclosed in Japanese Patent Laid-Open No. 2008-72377, a first high-frequency signal is generated with use of an OG signal, and a second high-frequency signal is generated with use of a luminance signal that has been generated from signals of all colors of pixels using an angle-adaptive SWY method that limits the bandwidth in not only the horizontal and vertical directions, but also the diagonal directions. Weighted-addition is then performed on the first high-frequency signal and the second high-frequency signal according to the spatial frequency of the signal to generate a third high-frequency signal, and the OG signal and the third high-frequency signal are added together, thus generating a final luminance signal.
However, when the technique disclosed in Japanese Patent Laid-Open No. 2003-348609 is applied to a signal obtained using a primary color Bayer pattern type of color filter array, there is the adverse effect in which a spurious resolution signal (spatial aliasing) is generated in the vicinity of the 45-degree and 135-degree directions when using an SWY signal generated by the luminance signal generation circuit shown in FIG. 21. This is thought to be due to the fact that in the circuit shown in FIG. 21, the bandwidth is limited in only the H direction and the V direction, and therefore the bandwidth is not sufficiently limited in the 45-degree and 135-degree diagonal directions.
Also, since an OG signal is used for a chromatic imaging subject, the resolution in the diagonal directions is not improved for chromatic imaging subjects. Furthermore, when an edge-emphasized signal is generated using a luminance signal in which portions (diagonal regions 2403) of the OG signal have been substituted with an SWY signal, portions where there is a switch between the OG signal and the SWY signal are emphasized, and an unnatural texture readily appears.
Also, in the technique disclosed in Japanese Patent Laid-Open No. 2003-196649, it is necessary to provide in advance and hold a plurality of interpolation filters corresponding to imaging subject angles. When the color space is set to, for example, NTSC-RGB, it is necessary to prepare an interpolation filter that maintains R:G:B=3:6:1, which is the luminance signal composition ratio. This results in the problem that the coefficient of the filter is limited, and it is impossible to perform filter processing that is optimal for the angle of the imaging subject. Similarly to Japanese Patent Laid-Open No. 2003-348609, since edge emphasizing is performed after mixing luminance signals computed using a plurality of interpolation methods, subtle switches between luminance signals computed using different methods become emphasized.
In the technique disclosed in Japanese Patent No. 3699873, an SWY signal is used in place of an OG signal only in the case of Mg and G imaging subjects, and therefore there is no improvement in the resolution of chromatic imaging subjects other than Mg and G imaging subjects. Furthermore, similarly to the techniques disclosed in Japanese Patent Laid-Open No. 2003-348609 and Japanese Patent Laid-Open No. 2003-196649, since edge emphasizing is performed after weighted-addition has been performed on luminance signals computed using a plurality of techniques, portions where there is a switch between methods become emphasized.
In Japanese Patent Laid-Open No. 2008-72377, image quality deterioration with a red imaging subject is suppressed, and therefore a red region is detected, and a final luminance signal is generated with use of a first high-frequency signal generated from an OG signal for that region. For this reason, there are problems such as the phenomenon in which a red imaging subject becomes blurred (in particular, blonde hair that is difficult to distinguish from red becomes blurred), and spatial aliasing of red diagonal lines cannot be eliminated.
The same problem occurs with blue imaging subjects depending on the spectral distribution characteristics of the color filter array.